When watching video a user can want to know more about what is being viewed. For example, a user may want to know more about the actors, the location where a scene is being filmed, the clothing being worn by the actors, the furnishings in a room, the vehicles, or any other item in or feature about a scene. Accessing this information is constrained because the video content is image-based whereas a search to access the information is conventionally text-based. Remembering what was viewed and then later finding out about the information using a text-based search is cumbersome and inconvenient.
Audio-based technologies are available that use content recognition or simply have websites that provide details about the video content. Although such methods for accessing information about video content can provide scene-level information or more general information about the video content, existing methods do not provide frame-level information. Frame-level information refers to content concerning an individual video frame. Users prefer instantaneous information about the video content they are viewing.
A number of technological and social developments provide a context and motivation for developing methods to access video content interactively. The use of mobile devices is becoming ubiquitous and multitasking in which mobile devices are used simultaneously while performing other activities such as viewing video is becoming second nature. The delivery of video content by over-the-top (OTT) methods such as streaming video is also increasing. It is estimated that over 65% of households in the United States have a streaming device and the number is expected to increase dramatically in the coming years. As a result, the video being viewed at home will increasingly be delivered by streaming video content providers and not by broadcast or cable networks. The ability to provide OTT content to households is being facilitated by access to broadband networks and delivery of broadband service directly to the home. The quality of streaming high-definition video to the home is already acceptable, and increasing broadband speeds and infrastructure deployment will only improve the service and viewer experience. Another interrelated trend is the increasing development and adoption of mobile commerce. Currently, Internet commerce is estimated to be only 10% of the total individual consumer commerce and only a small fraction of that is done through a mobile device. However, consumer transactions through mobile devices are being facilitated by mobile provider purchasing applications. As a result, consumers are becoming increasingly more comfortable with making purchases over a mobile device.
Improved methods for accessing and interacting with OTT content such as streaming media are desired.